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Introduction 


One  method  of  formally  incorporating  prior  opinion  and  partial  information  into  the 
estimation  of  latent  trait  models  is  the  Bayesian  approach.  The  implementation  of  this 
approach  to  practical  problems  is  made  difficult  due  to  the  lack  of  tools  to  quantitatively 
deal  with  prior  information.  The  purpose  of  this  paper  is  to  examine  some  tools  to 
facilitate  the  selection  of  prior  distribution  and  to  demonstrate  how  this  distribution  can  be 
used  to  estimate  models  for  mental  testing. 

It  will  be  assumed  that  the  responses  to  test  items  are  dichotomous  (correct  or 
incorrect)  and  that  each  item  can  be  characterized  by  an  item  response  curve,  a  function  of 
ability  indexed  by  unknown  parameters,  called  item  parameters.  Most  of  the  discussions 
will  be  on  the  three-parameter  logistic  (3PL)  curve  (Birnbaum,  1968),  with  the  focus  on 
formulating  a  prior  distribution  and  on  computing  the  posterior  mode  of  item  parameters. 
The  assumptions  and  techniques  closely  follow  those  in  Tsutakawa  &  Lin  (19S6),  where  an 
illustration  was  given  for  the  two-parameter  logistic  (2PL)  with  a  prior  which  differs  from 
the  one  in  this  paper. 

A  standard  method  of  estimating  ability  and  item  parameters  for  3PL  is  maximum 
likelihood  (ML).  This  approach  has  been  extensively  discussed  by  Lord  (1980)  and  become 
quite  popular  since  the  availability  of  a  number  of  convenient  computer  programs  such  as 
LOGIST  (Wingersky,  Barton  &  Lord,  1982).  Under  the  assumption  that  the  abilities  are 
randomly  sampled  from  some  population  distribution,  the  marginal  maximum  likelihood 
(MML)  estimation  of  item  parameters  has  been  discussed  by  Bock  and  Aitkin  (1981)  and 
Ridgon  &  Tsutakawa  (1983),  among  others. 

Following  the  argument  developed  for  hierarchical  linear  models  by  Lindley  & 

Smith  (1971),  Swaminathan  &  Gifford  (1986)  have  proposed  the  use  of  the  joint  posterior 
mode  of  ability  and  item  parameters  for  3PL.  Mislevy  &  Bock  (1985)  and  Tsutakawa  & 

Lin  (1986)  use  the  EM  algorithm  (Dempster,  Laird  &  Rubin,  1977)  to  compute  the 
posterior  mode  of  item  parameters  for  3PL  and  2PL,  respectively.  The  priors  used  by 
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Swaminathan  &  Gifford  and  Mislevy  &  Bock  assume  independence  among  item 
parameters,  not  only  between  but  within  items.  In  Tsutakawa  &  Lin,  the  dependence 
among  parameters  within  items  is  introduced  into  the  prior  through  the  use  of  an  ordered 
bivariate  beta  distribution  for  values  of  the  item  response  curve  at  two  ability  levels. 
Mislevy  (1986)  proposed  representing  such  dependence  via  multivariate  normal  priors  on 
the  item  parameters.  The  proper  representation  of  the  joint  prior  distribution  of 
parameters  within  items  is  particularly  important  in  the  presence  of  preliminary 
information  or  previous  data  on  the  items. 

In  this  paper,  the  Tsutakawa  &  Lin  prior  is  modified  by  replacing  the  ordered  beta 
by  an  ordered  Dirichlet.  The  advantage  of  this  prior  is  that  it  has  few  parameters  and 
simpler  to  select  since  the  marginal  distributions  are  betas.  The  Dirichlet  distribution  has 
been  extensively  studied  in  Wilks  (1962)  and  used  as  prior  distribution  by  Ramsey  (1972) 
for  quantal  response  functions  in  bioassay  and  Ferguson  (1973)  for  nonparametric  inference. 
The  use  of  this  distribution  here  is  more  limited.  It  is  used  to  facilitate  incorporating  prior 
information  about  items  which  may  be  more  readily  available  in  terms  of  response 
probabilities  rather  than  item  parameters. 

The  paper  begins  with  a  statement  of  the  general  problem  and  a  discussion  of  some 
difficulties  encountered  in  implementing  Bayesian  principles.  To  facilitate  the  selection  of 
prior  distributions  a  reparameterization  of  the  item  parameters  is  introduced.  This  is 
followed  by  an  examination  of  the  Dirichlet  distribution  as  a  means  of  specifying  the  prior. 
The  method  is  adapted  to  estimating  3PL  curves  for  a  1987  American  College  Testing 
Program  (ACT)  math  test,  with  prior  distribution  based  on  a  distribution  of  3PL  curves 
from  a  1981  ACT  math  test.  The  robustness  of  the  choice  of  prior  is  illustrated  in  terms  of 
changes  in  the  posterior  modes  as  the  amount  of  weight  placed  on  the  prior  is  varied. 

These  estimates  are  numerically  compared  to  MML  and  LOGIST  estimates.  A  plot  of  the 
estimated  3PL  curves  shows  how  the  Bayes  esitmates  are  shrunk  towards  the  prior  mean 
relative  to  either  MML  and  LOGIST.  This  has  the  effect  of  preventing  the  occurence  of 
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extreme  outcomes  frequently  encountered  by  maximum  likelihood  methods.  The  extensive 
computational  expressions  required  to  implement  the  EM  algorithm  are  summarized  in  the 
Appendix. 


General  Setup  and  Problems 

Consider  each  of  n  examinees  responding  to  a  test  with  k  items.  Let  y-  =  0  or  1 
according  as  the  response  to  item  j  by  examinee  i  is  incorrect  or  correct.  Assume  the 
probability  of  a  correct  response  to  an  item  is  given  by  an  item  response  function  p^(0) 
depending  on  the  unknown  item  parameter  £  and  real  valued  ability  0.  For  3PL,  which 
will  be  discussed  below,  this  function  has  the  form 


=  c  +  l+exfl-a( 


for  -oo  <  9  <  oo,  where  £  =  (a,  b,  c),  0  <  a,  -co  <  b  <  oo,  and  0  <  c  <  1.  The  parameter 
space  for  £  will  be  denoted  by  Cl. 

Given  k  items  with  parameters  £  =  (£p— >£].)  and  n  individual  with  abilities 
0=  (0p...,0n),  assume  conditional  independence  among  the  responses  so  that  the  joint 
probability  of  the  n*k  matrix  y  =  ((y^))  is  given  by 


P(y|£,0)  =  n  n  Ptyjjl^i), 
i  j  J  J 


where 


P(yjjkj^i)  =  -  p^j)}  y^.  Yjj  =  0,1. 


Moreover  assume  that  O^,...,0n  are  iid  N(0,1).  [Without  loss  of  generality,  N(0,1)  is  used 
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rather  than  N(/q<7  )  with  (/j,,a  )  unknown  in  order  to  avoid  the  indeterminacy  in  the 
parameterization  associated  with  the  3PL  model  (Lord,  1980,  pp.  36-38).] 

The  problem  is  to  estimate  f  based  on  the  data  y  when  there  is  previous  information 
about  the  items.  If  this  information  can  be  summarized  in  terms  of  a  prior  pdf  p(f)  of  £, 
Bayesian  principles  suggest  that  one  should  consider  the  marginal  posterior  distribution  of 
6  given  by 

p(f|y) «  p(f)n  f  n  P(yii|fi,tfi)tf<tfi)d«  (3) 

i  j  j  j  j 

where  y?(^)  is  the  N(0,1)  pdf. 

Having  accepted  this  general  principle,  the  major  obstacles  to  carrying  out  the 
Bayesian  approach  are  two-fold.  The  first  obstacle  is  the  selection  of  the  prior  or  p(f). 

The  second  is  carrying  out  the  computation  in  order  to  summarized  the  information  about 
£  after  observing  y.  With  the  influx  of  high  speed  computing  and  the  availability  of 
numerical  approximations,  the  second  problem  is  becoming  less  crucial,  though  far  from 
solved.  The  solution  to  the  first  problem  is  largely  subjective  and  not  adequately 
discussed.  This  paper  is  primarily  on  techniques  for  dealing  with  the  first  problem. 

Reparameterization  for  3PL 

At  three  fixed  ability  levels  <  t^  consider  the  values  of  the  3PL  curve 


P(0  =  (PrP2,P3), 


(4) 


where 


1— c 


Pj  =  c  +  f^.exppa(t._bj  j  1  =  1*2,3.  Now  consider  the  space  9  spanned  by  p(£), 
i.e.,  9=  (p]p  =  p(£)  for  some 
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The  Jacobian  of  this  transformation  is  defined  by  the  determinant 


J  = 


dpj/da  dp^/ &o  dp^/dc 
dp2/da  dp2/<9b  dp^dc  > 
dpg/da  <9p3/5b  dp^j dc 


(5) 


which  may  be  simplified  to 

2  3 

J  =  a(l-c)  exp(ab)(t3~t1)  n  ^  ^  {w^exp^at^)  +  w23exp(-at1)  -exp(-at2)},  (6) 

where  w12  =  ( t2— t j ) / ( t3~ t j ) ,  w23  =  ( 1 3~ 1 2 ) / 1 3~ t ^  =  [1  +  exp{-a(t-b)}]-1  and  V>t 

=  1  —  (pt  .  The  fact  that  J  >  0  for  all  (a,b,c)c  0,  may  be  seen  by  noting  that  all  factors 
li 

outside  the  brackets  {}  are  positive  and  that  the  expression  in  {}  is  positive  since  the  sum 
of  the  first  two  terms  is  a  weighted  average  of  values  of  a  convex  function  which  is  larger 
than  the  third  term,  the  value  of  this  function  at  t9  (the  weighted  average  of  t^  and  t3).  It 
follows  that  the  transformation  (4)  is  nonsingular  and  the  3PL  curves  may  be 
parameterized  in  terms  of  pe9. 

Now  let  0  denote  the  set  of  all  triples  p  =  (PpP2,P3)  with  0  <  p^  <  p9  <  p3  <  1. 
Inspite  of  the  richness  of  the  3PL  family,  not  all  points  in  0  belong  to  9  The  following 
result  is  useful  in  characterizing  the  points  in  9. 

Theorem.  Given  t^  <  t2  <  t3  and  any  p tO,  there  exists  a  point  such  that  p  =  p(£)  if 
and  only  if  there  exist  some  c  such  that  0  <  c  <  p^  and 

( L2 — L 1 ) / (t2  1 1 )  =  (L3  L2)/ 1^  t2),  (7) 

where 


Lj  =  ln{(p- — c)/ (1 — Pj)} ,  i=l,2,3, 


(8 


Proof:  Given  (a,b,c)  e  ft,  the  equation  (4)  may  be  rewritten 


Lj  =  a(tj-b),  i 


1,2,3. 


(9 


But  (9)  implies  (7)  and  (4)  implies  0  <  c  <  p, . 

Conversely,  given  (p^,  P2,  Pg)f^  suppose  there  exists  a  c  such  that  0  <  c  <  p^  and 
(7)  holds.  For  this  c,  a  and  b  may  be  solved  from  (9)  and  are  given  by 


a  —  (L^  Lj)/(t<jt^), 
b  =  tj  —  Lj/a  . 


(10 


Note  that  a  >  0  since  c  <  p^  <  P2  and  >  L^.  Thus  the  resulting  £  =  (a,b,c)e  ft  and 
P(0  =  P 

An  important  special  case  in  which  one  can  explicitly  find  f  for  a  given  pe.55  is 
where  the  t.  are  equally  spaced.  The  solution  is  given  by  the  following. 


Corollary.  Given  p i9  and  t^— tg  =  t2— h  >  c  IS  §'ven  by 

-  -  -0MI32- 4tt7)1/2 
c  “  2a 


(11 


o-  (1— Px)(l— P3)  —  (1— P2)2’ 

0=  2p2(l-P1)(l-P3)  +  (P1+P3)(l-P2)2- 
7  =  p|(l— Px)(l— P3)  ~  PiP3(l— P2)2- 


where 
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Proof.  If  t3~t2  =  t^-tj  >  0,  (7)  simplifies  to 

(p2-c)2/(i— P2)2  =  {(pL— c)/(l— P1)}{(P3— c)/(l— P3)}  (12) 

This  is  a  quadratic  equation  in  c  whose  solution  is  given  by  the  Corollary.  It  is 
immediately  seen  by  inspection  that  one  of  the  roots  is  always  equal  to  c=l  and  may  be 
ignored  since  we  must  have  c  <  p1  <  1.  Once  c  is  available  a  and  b  may  be  obtained  from 
(10). 

As  an  example  of  a  point  p  in  0  not  in  9,  consider  (t, ,  t9,  t.j)  =  (-1,  0,  1)  and 
(pp  P2>  P3)  =  (-05,  -50’  -55)-  In  this  case  (<*’  &  7)  ~  (-1775,  -.2775,  .0100)  and  c  =  .56 
and  1.  Since  c  >  pppg.2 

Constrained  Dirichlet  Prior 

Following  Tsutakawa  &  Lin  (1986),  consider  prior  information  about  the  item 
response  curves  at  fixed  ability  levels  rather  than  about  item  parameters  directly.  Then 
consider  the  distribution  induced  on  the  item  parameters  through  the  transformation 
relating  values  of  the  item  response  function  to  the  item  parameters. 

For  a  given  item  let  Pp—»Pm  be  the  probabilities  of  correct  responses  at 
predetermined  ability  levels  t1  <  t2  Now  define  the  increments 

X1  =  pr 

x2  =  P  9  p 1 ’ 

.  “  (13) 

xm+ 1  ~  ^  pm 

Suppose  our  prior  information  about  these  increments  can  be  represented  by  the 
m  —  dimensional  Dirichlet  distribution  with  pdf, 


PH 


p  /  xt  \  J,N— 1  7T0N— 1  7T  .  ,N— 1 

f(xlv"’xm)  =  m+1  X1  x2  ~  -(l-xr--xm)  m  (14) 

n  r  ( x  n) 

s  =1  s 


0  <  x,  <...<  xm  <  1,  where  (7rp--->7rm!  N)  are  parameters  such  that  7rg  >  0,  £  t  =  1, 

s  =1 

and  N  >  0.  The  first  two  moments  of  (14)  are  given  by 


E(xs>  =  V 


Var (xg)  =  7rs(l-7rs )  /(N+l), 


Cov(xr  ,xg)  =  -7rr^s/(N+l  )  •,  for  r^s. 

Under  the  transformation  (4),  (pp...,pm)  has  the  ordered  Dirichlet  distribution 
(Wilks  1962)  with  pdf 


r(N)  TTiN-l  t9N-1 

8(pl’-’pm)  =  m+1  P1  ^p2_pl^  "^1_pm^  (16) 

n  r(rN) 

S=1 


for  0  <  <...<  Pm<l-  Because  of  the  well  known  "lumping"  property  of  the  Dirichlet,  pg 

=  Xj+...+xg  will  have  a  marginal  distribution  which  is  beta.  From  (15),  the  first  two 
moments  of  (pp...,pm)  are 

»S  =  E(PS)  =  *!+...  +  V 

(17) 

^  =  Var(pa)  =  /ts(l-//s)/(N+l), 


for  1  <  s  <  m,  and 
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<TIS  =  Cov(pr,pg)  =  /ig(l— //r)/(N+l)  -Pr(/*S-Pr)/(N+1), 


for  1  <  r  <  s  <  m. 

Now  the  specification  of  /xg  <  /xg+p  will  uniquely  define  the  value  for 

(7Tp...,7rm,N)  and  hence  the  joint  distribution  for  (Pi>--->Pm)-  In  practice  the  fi's  can  be 

chosen  to  represent  the  prior  point  estimate  of  the  p's  and  N  to  represent  the  weight  of  the 

o 

prior  or  tightness  of  the  prior  about  the  estimate  as  expressed  in  the  variances  a  .  Further 

s 

discussion  on  selection  will  be  given  in  the  example. 

For  the  application  to  3PL,  consider  the  case  m=3  and  the  constrained  distribution 
with  pdf 


p  /  \  4  X  N  1 

gc(PrP2>P3)  =  K  -5 — 5—!—  npss  ,  (18) 

nr(iN)s4 

S—l  s 

for  (ppP2,Pg)e^’  and  0  otherwise,  where  K  is  a  normalizing  constant.  (See  Box  &  Tiao, 
1973,  p.  67  for  a  general  discussion  on  constrained  distributions.)  This  distribution  induces 
a  distribution  for  through  the  transformation  (4).  The  induced  distribution  will  have 
pdf 


p(a,b,c)  =  |Jjgc(p1(0,  P2(0>  P3(0) 

for  where  J  is  defined  by  (6)  and  (p^),  p9(<f),  P3(0)  =  p(0- 

There  is  some  problem  in  selecting  (tTj,  x.v  x^,  x^,  N)  since  the  moments  under  g^ 
will  differ  from  those  under  g.  This  will  not  be  an  important  issue  when  N  is  large  since 
the  continuity  of  the  transformation  assures  us  that  a  tight  distribution  for  p  will  induce  a 
tight  distribution  for  £  and  the  discrepancy  between  g  and  g^  will  be  negligible.  The 


numerical  work  in  the  next  section  suggests  that  one  may  approximate  g^  by  g  for 
moderate  size  N. 

Formulating  a  Prior  from  Previous  Data 

The  application  of  previous  information  to  analyze  new  data  will  be  illustrated  here 
and  in  the  next  section  in  term  of  two  ACT  math  tests.  The  data  from  the  1981  test  will 
be  use  to  form  a  prior  distribution  for  item  parameters  used  to  estimate  3PL  curves  for  the 
1987  test. 

Figure  1  gives  40  3PL  curves  estimated  by  LOGIST  based  on  a  random  sample  of 
n=2000  from  the  1981  test.  Considering  the  year  to  year  similarity  in  ACT  tests  and  the 
limited  information  about  the  individual  items  in  the  1987  test,  it  seems  reasonable  to 
assume  that  the  sample  characteristics  of  3PL  curves  for  1987  will  be  similar  to  those  for 
1981.  One  might  reason,  in  this  case,  that  the  1987  curves  will  behave  like  a  random 
sample  from  the  same  distribution  of  curves  that  produced  those  for  19S1.  If  additional 
information  is  available  for  specific  items  the  random  sample  assumption  would  be 
unreasonable  and  different  priors  should  be  formed  for  different  items.  There  is  nothing 
that  precludes  the  use  of  subjective  opinion  at  this  stage. 


In  order  to  select  the  parameters  for  the  Dirichlet  distribution,  the  three  levels 
chosen  for  this  example  are  (tp  t2,  t3)  =  (-1,28,  0,  1.28)  corresponding  to  the  (10,  50,  90) 


percent  points  of  the  normal  distribution.  The  sample  averages  pj  and  standard  deviations 


Sj  of  the  values  of  the  curves  in  Figure  1  at  these  three  point  are  tabulated  in  Table  1. 

Consider  matching  these  moments  to  the  moments  (17)  of  the  ordered  Dirichlet. 
Since  the  variances  cannot  be  matched  at  the  three  levels  by  a  single  N,  the  average  of  the 
three  N's  obtained  from  three  separate  Fittings  is  used.  The  average  so  computed  is 
N  =  (16.2  +  6.6  +  5.6)/3  =  9.8.  The  3PL  curve  passing  through  the  points  (tpi^), 

and  the  resulting  marginal  beta  distributions  of  Pp  p9,  p~  are  plotted  in 


Figure  2. 

The  relative  position  of  9  to  0  for  this  example  is  sketched  in  Figure  3.  In  order  to 
relate  the  unconstrained  to  the  constrained  Dirichlet,  points  in  0  were  randomly  simulated 
and  tested  for  inclusion  in  9  by  the  criterion  stated  in  the  Theorem.  Of  the  1,000  points 
thus  simulated  81%  were  also  in  9  and  had  the  sample  means  and  standard  deviations 
tabulated  in  Table  1.  The  discrepancy  between  the  distribution  in  9  and  0  averages 
about  4%  in  terms  of  the  means  and  5%  in  terms  of  the  standard  deviations. 

In  order  to  examine  the  effect  of  N  on  the  moments,  the  simulation  was  repeated  for 
N=3  and  24.  Of  the  1,000  points  70%  were  in  9  for  N=3  and  88%  for  N=24.  The  sample 
results  in  Table  1  indicate  the  increased  similarity  between  the  Dirichlet  and  constrained 
Dirichlet  as  N  increases.  The  results  also  suggest  the  possibility  for  finding  a  value  of  N  for 
which  the  sample  moments  obtained  from  the  ACT  curves  will  be  reasonably  matched  to 
those  of  the  constrained  Dirichlet.  It  will  be  shown  in  the  next  section  that  the  estimated 
3PL  curves  are  quite  robust  with  respect  to  the  choice  of  N  and  that  a  precise  specification 
of  N  is  not  crucial. 

Posterior  Mode  for  1987  ACT 

The  prior  based  on  the  1981  test  will  now  be  used  to  estimate  k=40  3PL  curves  for 
the  1987  test  with  a  random  sample  of  n=400. 

The  item  parameters  will  be  changed  to  £  =  (b,c,d)  where  d  =  log  a  or  a  =  exp(d) 
in  (1).  This  reparameterization  is  made  to  enhance  the  asymptotic  normality  of  the 
posterior  distribution  and  to  speed  up  the  convergence  of  the  EM  algorithm.  (Similar 
strategies  have  been  suggested  by  Naylor  &  Smith,  1982  and  by  Mislevy,  1986.) 

The  posterior  mode  and  marginal  maximum  likelihood  (to  be  denoted  by  MLF3  for 
3PL)  estimates  of  the  item  parameter  £  were  computed  via  the  EM  algorithm. 
Computational  expressions  are  summarized  in  the  Appendix.  Figures  4,  5,  and  6  give 
scatter  plots  of  the  b,c,  and  d  parameters.  The  two  estimates  of  the  b  parameter  are 


generally  quite  close  except  in  the  lower  range  where  the  Bayes  estimates  shows  more 
shrinkage  toward  the  average.  This  type  of  shrinkage  is  more  pronounced  for  the  c 
parameters,  where  5  items  had  c  parameters  which  were  positive  under  Bayes  but  zero 
under  MLF3.  The  estimates  of  the  d  parameters  were  also  fairly  close  except  for  the  3 
items  with  large  d  values  for  MLF3. 

To  study  the  effect  of  the  prior  on  the  posterior  mode  two  additional  cases 
corresponding  to  N=3  and  24,  discussed  in  the  last  section,  were  considered.  Figures  7 
through  10  show  a  sample  of  estimated  3PL  curves  under  MLF3,  LOGIST  and  Bayes  for 
N=3,  9.8,  24.  They  also  show  the  prior  mean  to  illustrate  the  amount  of  attraction 
towards  the  prior  means  as  a  function  of  N. 

Item  27,  shown  in  Figure  7,  exhibits  a  pattern  where  all  estimates  are  mutually 
close  and  close  to  the  prior  mean.  Item  7,  shown  in  Figure  8,  exhibits  a  more  typical 
pattern,  shared  by  most  items,  where  the  estimated  curves  are  fairly  close,  but  with  the 
Bayes  estimates,  particularly  those  with  N  large,  being  closer  to  the  prior  mean  than  either 
MLF3  or  LOGIST.  Item  13,  shown  in  Figure  9,  is  the  case  showing  the  largest 
discrepancy,  particularly  with  respect  to  slope.  The  instability  of  the  LOGIST  estimate 
was  indicated  by  the  low  value  of  b— 2(a/1.7),  which  was  given  as  —13.01  for  this  item.  The 
instability  appears  to  be  due  to  the  difficulty  of  the  item  which,  in  turn,  caused  a 
considerable  amount  of  guessing.  Item  22,  shown  in  Figure  10,  exhibits  a  moderately  close 
agreement  for  the  central  and  upper  6  values,  although  there  are  notable  differences  among 
the  slopes  and  lower  asymptotes. 

Although  it  is  difficult  to  make  general  statements  based  on  the  limited  data 
studied  here,  the  results  confirm  certain  properties  found  in  related  studies.  There  is 
general  agreement  among  the  estimates  for  most  items.  When  disagreement  exists  it  tend 
to  occur  where  non— Bayesian  estimates  take  on  extreme  values.  The  choice  of  N  is  not  too 
essential  and  appears  more  crucial  in  situations  where  the  non— Bayesian  estimates  are  least 
stable. 


Discussion 


The  primary  purpose  of  this  paper  has  been  to  demonstrate  the  Bayesian  use  of 
previous  information  to  analyze  new  item  response  data.  Discussions  on  the  advantages 
and  disadvantages  of  Bayesian  methods  may  be  found  in  Lord  (1986),  Mislevy  (1986), 
Swaminathan  k  Gifford  (1986)  and  Tsutakawa  k  Lin  (1986)  and  will  not  be  repeated  here. 

The  purpose  of  working  through  the  Dirichlet  is  to  facilitate  the  formulation  of  a 
prior  distribution  in  terms  of  measurements  more  familiar  to  the  user.  Although  the 
illustration  here  has  been  limited  to  an  exchangeable  prior,  the  method  may  be  easily 
modified  to  situations  where  prior  information  varies  from  item  to  item.  This  would 
include  instances  where  items  are  assembled  from  several  sources,  including  those  where 
items  have  been  previously  analyzed  under  different  models,  e.g.,  2PL. 

One  technical  problem  associated  with  the  current  approach  is  the  difference 
between  the  space  9  spanned  by  the  3PL  curves  and  the  space  0  of  the  ordered  Dirichlet. 
Although  this  is  not  a  problem  when  N  is  large  ,  some  rule  of  thumb  adjustment  would  be 
desirable  for  small  N.  The  robustness  of  the  posterior  mode  suggests  that  a  precise  value 
will  not  be  unnecessary. 

One  limitation  of  the  Dirichlet  prior  is  that  it  has  only  one  parameter,  N,  to  express 
the  tightness  of  the  prior  distribution.  In  cases  where  there  is  considerable  variability  in 
information  from  one  tj  to  another,  the  use  of  the  ordered  beta  (Tsutakawa  k  Lin,  1986) 
would  seem  preferable.  This  situation  could  arise,  for  example,  when  items  have  been 
previously  used  on  a  group  of  individuals  whose  ability  levels  are  generally  lower  (or 
higher)  than  those  of  the  current  examinees. 

The  use  of  the  prior  distribution  here  was  limited  to  obtaining  the  posterior  mode,  a 
point  estimate  of  the  item  parameter.  A  more  important  use  would  be  to  analyze  the 
posterior  uncertainty  in  both  the  item  and  ability  parameters.  Such  uncertainties  in  the 
item  parameters  can  be  evaluated  by  the  posterior  covariance  matrix,  which  can  be 


Appendix:  Computational  Expressions  for  the  EM  Algorithm 
As  shown  in  Tsutakawa  and  Lin  (1986),  the  key  steps  of  the  EM  algorithm  in 
finding  the  posterior  mode  of  f  may  be  summarized  as  follows. 

Starting  with  some  initial  approximation  £ 0  to  the  mode,  maximize  separately  for 


each  j  the  functions 


n  r 

•S  1 

i=lJ 


log  P(yij|^i^j)p(^ly5^0)d^i  +  log  p(£p, 


(A.l) 


where  y j  =  (y^ p(0j|yj,£°)  is  the  posterior  pdf  of  9 ■  when  £  is  known  and  equals  f°, 
and  p(^j)  is  the  prior  for  the  jth  item  parameter,  j  =  l,...,k.  Then  iterate  the  maximization 
of  these  function  after  replacing  £°  by  the  value  of  f  =  Up--^)  which  maximized  the 
function  at  the  last  iteration.  The  iteration  is  repeated  till  some  convergence  criterion  is 
satisfied. 

The  maximizations  require  numerical  integration  and  some  iterative  procedure  such 
as  the  one  by  Marquardt  (1963),  which  is  used  here.  Marquardt's  procedures  requires  the 
evaluation  of  the  first  and  second  partial  derivatives  of  (A.l)  with  respect  to  =  (bj,Cj,dj). 

Since  the  maximization  is  carried  out  separately  for  each  j,  the  subscript  j  will  be 


dropped  and  notations 


Z  =  sjlog  P(yij|«i,{j)p(«i|yi,{°)dft1 


g  (i,0)  =  log  Pfyjjl^j) 

will  be  used.  Now  denote  the  first  and  second  partials  of  Z  and  g(i,0)  with  respect  to 
(b,c,d)  =  (bj,Cj,dj)  by  Zu,Zuy,  gu(i,0)  and  guy(i,0)  for  u,v  =  1,2,3  so  that,  for  example, 
Z2  =  dZ/dc  and  gi;j(i,0)  =  c? g(i,9)/dbdd. 


"rrrv  ”  wwu.’wyww  wTrrrrr7Tr. 


To  simplify  the  notation  define 
=  {1  +  exp[-exp(d)(0-b)]}-1, 

^=1~^ 

A^=  {1  +  c  exp[-exp(d)(#-b)]}-1, 

Vq  =  {c  +  exp[exp(d)((  0-b)]  }_1 . 

Then  the  derivatives  of  g(i,0)  may  be  expressed  as  follows. 


giM  =-exp(d)(yi-Ar^), 
g2M  =  yjj[^-  l/(c— 1)]  +  l/(c-l), 

g3(i,/?)  =  (0-b)exp(d)(yij.A^-^), 
gn(i,^)  =  exp(2d){cyijA^¥?^}, 
g12(i^)  =  y^xpld)^^, 

g13(i,<?)  =  -exp(d)(yjjA^-^)  +  exp(2d)(f-b)[^^-eyi.A^0], 

g22O>0)  =  ~fyij4  +  C1— yij)/(c— 1)2]^ 

g23(i^)  =  -exp(d)(^-b)yjjA^ 

g33O,0)  =  (0-b)2exp(2d){cyjjA^^^}  +  (^-b)exp(d){yjjA^-^}. 


The  derivatives  of  Z  are  given  by 


and 

! 


I 


where 


) 

i 

I 


n 

Zu  = 

Zuv  = 


rOO 


gu(i)=  gu0,%(%p£  )d9 


and 


1 


1 


guv0)  =  ]  guv(i^)p(^yi^°)d^ 


See  Tsutakawa  (1984)  for  scaling  techniques  and  Gauss— Hermite  approximations  of  these 
intergrals. 

Again  suppressing  the  subscript  j,  the  prior  for  the  jth  item  parameter  is  given  by 


V-t — 1  24,-1  V.—l 

p(b,c,d)  *  (Jlpj  (p2— Pi)  (P3-P2)  ^  (H^)  4 


where  v.  = 


pi  “  c  +  1+exp 
for  i  =  1,2,3  and  J  is  the  Jacobian  given  by 


J  =  exp(2d+bed)(l-c)2  II  {<p  ±  ) 

i=l  li  li 

{(t3-t2)exp(-t1ed)+(t2-t1)exp(-t3ed)-(t3-t1)exp(-t2ed)}. 


J  can  be  shown  to  be  positive  and  its  expression  differs  form  (6)  since  the  parameterization 

is  different.  When  t2  =  0,  which  is  the  case  used  in  the  numerical  examples,  there  is  a 

slight  simplification  in  the  expressions  for  the  log  prior  and  its  first  two  partial  derivatives 

which  may  be  given  as  follows. 

log  p(b,c,d)  =  constant 

j  3 

+  2d  +  bea  +  21og(l— c)  +  E(logpt  +Iog0t )  +  log{tj(l-f3)  -t^l-fj)} 

4 

+  £(^-1)  iog(Pj  — pi—1) , 

where  fj  =  exp(-edtj),  for  i  =  1,3,  pg  =  0,  and  p^  =  1. 

,  3  .  4 

<91og  p(b,c,d)/flb  =  ea  +  E  ea(</>,  -A  )  +  E(t/.-l)aog(p.-p.  ,)/5b, 

i=l  \  li  i=l  1  111 


L*Jk,U 


i*  ft**  J 


flog  p(b,c,d)/ do.  =  2/ (c — 1)  +  ^(i/.-ljaoglPj-p.J/ac 

i=l  1  111 

.  edt,tq(L-f1)  3  , 

ai _ j  \  /aj _  a  i  l~u  ,  i  O  o  1  i  r> 


dlog  p(b,c,d)/dd=  2  +  be  -f 


“fr  +  ^  e  (tj-b)(^t  —<p.  ) 
V  i=l  1  li  li 


+  (i/.-l)aog(pi-pi__1)/ad 

^log  p(b,c,d)/5b2  =  - 2e2d  E  <pt  A  +  E  (MjA^Pj-P:  ,)5b2, 

i=l  li  i=l  1  111 

^log  p(b,c,d)/ (XT  =  -  2/(1 — c)2  +  E(i/.-l)a2log(pi-p.  1)/^c2, 

i=l  1  111 

^log  p(b,c,d)/dd2  =  bed 

e  ^3e  )— )}{^(1— - ^3)“^ 3(1— f  1 ) }  (e  ^  1  ^3(^1  ^3) } 


{t1(l-f3)  -t3(l-f1)}A 

+  £  {ed(tj-b)(^ t-v>t  )  -2e2d(t.-b)V  \  }  +  E  (i'.-l)Aog(p.-p.  ,)/ad2, 
i=l  1  i  1  li  i=l  1  111 

^log  p(b,c,d)/<9b&  =  E  (i/.-l) ^logfo-p.  ,  )/dbdc, 

i=l  1  111 

3 

d^log  p(b,c,d)/<9bdd  =  ed  +  E  {ed(<^  -ipt  )  4-  2e2d(t;-bW  ip.  } 

i=l  li  li  1 

+  E  (t/rl ) ^logCpj-pj  )/5b^d, 
i=l  1  111 

5^1og  p(b,c,d)/ dcdd  =  £  (i/.-l)flPlQg(p.-p.  .J/flccH. 

i=l  1  111 

In  order  to  complete  the  computational  expressions  for  the  derivatives  of  the  log 
prior  the  first  two  derivatives  of  Iog(p- —  Pj_^),  are  needed.  Using  the  notation 
/?,  =  b,  /?2  =  c,  /?2  =  d,  these  derivatives  are  given  by 


TO-*og(pi-p._1)  -  [•57r-M7i]/(PrPi_i)> 
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r,s  =  1,2,3.  For  i=0  and  4  the  derivatives  of  Pj  are  zero.  For  i  =  1,2,3,  they  are 

dp.Jdb  =  — ed(l-c 

dp.Jdc  =  1  -  <pt 

dp.Jdd  =  ed(l-c)(ti-b)v?t  , 

fiv-Jdb2  =  e2d(l-c){^t>t.  -  ^  }, 

d 2  Pj/^c2  =  0, 

fip.Jdd2  =  ed(l— c)(tj— b)v?t  1>t  {l  +  ed(trb)(^t  -</>t )}. 

(fipjdbdc  =  ed</?.  ip.  , 

1  li  li 

d2 p.Jdhdd  =  -ed{l-c)<pt  ipt  {1  +  ed(tj-b)^t  -  ed(tj-b)0t  }, 
d 2 p.JdcdA  =  —  ed(t-b)0t  ip^  . 
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TABLE  1 

Summary  of  Dirichlet  (D)  and  Simulated  Constrained  Dirichlet  (CD)  Priors 


s 


2 


Distribution 


N 


P1  p2 


FIGURE  4 

ALF3  estimates  of  b 


FIGURE  8 

Estimated  3PL  curves  for  item  7 


FIGURE  9 

Estimated  3PL  curves  for  item  13 


FIGURE  10 

Estimated  3PL  curves  for  item  22 
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Rekruter i ngs-En  Se I ec t i ecentrum 

Kwartier  Koningen  Astrid 

Bru i jnstraat 

1120  Brussels,  BELGIUM 

Dr.  Robert  Breaux 
Code  7B 

Naval  Training  Systems  Center 
Orlando,  FL  32313-7100 

Dr.  Robert  Brennan 
American  College  Testing 
Programs 
P.  0.  Box  168 
Iowa  City,  I A  52243 

Dr.  James  Carlson 
American  College  Testing 
Program 
P.0.  Box  168 
I owa  City,  IA  52243 

Dr .  John  8 .  Carro I  I 
409  Elliott  Rd.,  North 
Cnape I  Hill,  NC  27514 


Dr.  Menucha  Birenbaum 
School  of  Education 
Tel  Aviv  University 
Ramat  Aviv  69978 
ISRAEL 

Dr.  Arthur  S.  Blaiwes 
Code  N712 

Naval  Training  Systems  Center 
Orlando,  FL  32313-7100 

Dr.  Bruce  Bloxom 

Defense  Manpower  Data  Center 

550  Camino  El  Estero, 

Suite  200 

Monterey,  CA  93943-3231 


Dr.  Robert  M.  Carroll 
Chief  of  Naval  Operations 
0P-01B2 

Washington,  DC  20350 

Dr.  Raymond  E.  Christal 
UES  LAMP  Science  Advisor 
AFHRL/MOEL 

Brooks  AFB,  TX  73235 

Dr .  Norman  Cliff 
Department  of  Psychology 
Un i v .  of  So.  California 
Los  Angeles,  CA  30033-1061 
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Oi rector, 

Manpower  Support  and 
Readiness  Program 
Center  for  Naval  Analysis 
2000  North  Beauregard  Street 
Alexandria,  VA  22311 

Dr .  Stan  ley  Co  I  I  ye  r 
Office  of  Naval  Technology 
Code  222 

800  N.  Quincy  Street 
Ar! ington,  VA  22217-5000 

Dr.  Hans  F.  Crombag 
F acu I  tv  of  Law 
University  of  Limburg 
P.0.  Box  616 
Maastr i cht 

The  NETHERLANDS  6200  MD 

Dr.  Timothy  Davey 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  C.  M.  Dayton 
Department  of  Measurement 
Statistics  &  Evaluation 
College  of  Education 
University  of  Maryland 
College  Park,  MD  20742 

Dr .  Ralph  J .  DeAv a  I  a 
Measurement,  Statistics, 
and  Evaluation 
Benjamin  Bldg.,  Rm.  4112 
University  of  Maryland 
Col  lege  Park,  MD  20742 

Dr.  Dattprasad  Divgi 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.Q.  8ox  16268 
Alexandria,  VA  22302-0268 

Dr.  Hei-Ki  Dong 
Bell  Communications  Research 
6  Corporate  Place 
PYA-1K226 

Piscataway,  NJ  08854 


Dr.  Fritz  Drasgow 
University  of  Illinois 
Department  of  Psychology 
603  E .  Dan i e I  St . 

Champaign,  IL  61820 

Defense  Technical 

Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 
(12  Cop i es> 

Dr.  Stephen  Dunbar 
224B  Lindquist  Center 
for  Measurement 
University  of  Iowa 
I owa  City,  I A  52242 

Dr.  James  A.  Earles 

Air  Force  Human  Resources  Lab 

Brooks  AFB,  TX  73235 

Dr.  Kent  Eaton 
Armv  Research  Institute 
5001  tisennower  Avenue 
Alexandria,  VA  22333 

Dr.  John  M.  Eddins 
University  of  Illinois 
252  Engineering  Research 
Laboratory 

103  South  Mathews  Street 
Urbana,  IL  61801 

Dr.  Susan  Embretson 
University  of  Kansas 
Psychology  Department 
426  Fraser 
Lawrence.  KS  66045 

Dr.  George  Englehard,  Jr. 
Division  of  Educational  Studies 
Emory  Un i vers  t  ty 
210  F i shbu rne  Bldg. 

Atlanta,  oA  30322 

Dr.  Benjamin  A.  Fairbank 
Performance  Metrics,  Inc. 

5825  Ca I  I aghan 
Suite  225 

San  Antonio,  TX  78228 
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Dr.  P  -A .  Federico 

Code  51 

NPRDC 

San  Diego,  CA  92152-6800 

Dr.  Leonard  Feldt 
L i ndqu i st  Center 
for  Measurement 
University  of  I owa 
Iowa  City,  IA  52242 

Dr.  Richard  L.  Ferguson 
American  College.  Testing 
P.0.  Box  168 
I owa  City,  I A  52243 

Dr.  Gerhard  Fischer 
L  i  eb  i  ggasse  5/3 
A  1010  Vi enna 
AUSTRIA 

Dr.  Myron  Fischl 
U.S.  Army  Headquarters- 
DAPE-MRR 
The  Pentagon 

Washington,  DC  20310-0300 

Prof.  Donald  Fitzgerald 
University  of  New  England 
Department  of  Psychology 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

Mr.  Paul  Foley 

Navy  Personnel  RAD  Center 

San  Diego,  CA  92152-6800 

Dr .  A  I f  red  R  Freg  I  y 
AFOSR/NL ,  Blag.  410 
Bolling  AFB,  DC  20332-6448 

Dr.  Robert  D.  Gibbons 
Illinois  State  Psychiatric  Inst. 
Rm  523W 

1601  W.  Taylor  Street 
Chicago,  IL  60612 

Dr.  Jan  ice  Gi f f ord 
University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01003 


Dr.  Robert  GI aser 
Learning  Research 

A  Development  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  A  34th  Street 
Baltimore,  MO  21218 

D0RN1ER  GMBH 
P.0.  Box  1420 
D-7990  Friedrichs ha fen  1 
WEST  GEPMANY 

Dr.  Ronald  K.  Hambleton 
University  of  Massachusetts 
Laboratory  of  Psychometric 
and  Evaluative  Research 
Hills  South ,  Room  152 
Amherst,  MA  01003 

Dr.  Delwyn  Harnisch 
University  of  Illinois 
51  Gerty  Drive 
Champaign,  IL  61820 

Dr .  Grant  Henn i ng 
Senior  Research  Scientist 
Division  of  Measurement 
Research  and  Services 
Educational  Testing  Service 
Princeton,  NJ  08541 

Ms.  Rebecca  Hetter 
Navy  Personnel  RAD  Center 
Code  63 

San  Diego,  CA  92152-6800 

Dr.  Pau I  W.  Ho  II  and 
Educational  Testing  Service,  2 1  — T 
Reseda  I e  Road 
Princeton,  NJ  08541 

Prof.  Lutz  F.  Hornke 
Institut  fur  Psychologie 
RWTH  Aachen 
Jaegerstrasse  17/19 
D-5100  Aachen 
WEST  GERMANY 
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Or.  Paul  Horst 
677  G  Street,  #184 
Chula  Vista,  CA  92010 

Mr.  Dick  Hoshaw 
OP- 135 

Ar I i ngton  Annex 
Room  2834 

Washington,  DC  20350 

Dr.  Lloyd  Humphreys 
University  of  Illinois 
Department  of  Psychology 
603  East  Daniel  Street 
Champaign,  IL  61820 

Dr.  Steven  Hunka 
3-104  Educ.  N. 

University  of  Alberta 
Edmonton,  Alberta 
CANADA  T6G  2G5 

Dr . . Huynh  Huynh 
Co  I  I ege  of  Educat i on 
Univ.  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Robert  Jannarone 
Elec,  and  Computer  Eng.  Dept. 
University  of  South  Carolina 
Columbia,  SC  29203 

Dr.  Douglas  H.  Jones 
Thatcher  Jones  Associates 
P.0.  Box  6640 
10  T  rat'  a  I  gar  Court 
Lawrencev  i  I  I e  ,  NJ  08648 

Dr.  Mil  ton  S.  Katz 
European  Science  Coordination 
Office 

U.S.  Army  Research  Institute 
Box  65 

FPO  New  York  09510-1500 

Prof.  Joan  A.  Keats 
Department  of  Psychology 
University  of  Newcastle 
N.S.W.  2308 
AUSTRALIA 


Dr.  G.  Gage  Kingsbury 

Portland  Public  Schools 

Research  and  Evaluation  Department 

501  North  Dixon  Street 

P.  0.- Box  3107 

Portland,  OR  97209-3107 

Dr.  William  Koch 
Box  7246,  Meas.  and  Eval.  Ctr. 
University  of  lexas-Austi n 
Austin,  TX  78703 

Dr.  James  Kraatz 
Computer-based  Education 
Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

Dr.  Leonard  Krouker 
Navy  Personnel  RXD  Center 
Code  62 

San  Diego,  CA  92152-6800 

Dr.  Jerry  Lehnus 

Defense  Manpower  Data  Center 

Suite  400 

1600  Wilson  B I vd 

Rosslyn,  VA  22209 

Dr.  Thomas  Leonard 
University  of  Wisconsin 
Department  of  Statistics 
1210  West  Dayton  Street 
Madison,  WI  53705 

Dr .  Mi chae I  Levine 
Educational  Psychology 
210  Educat ion  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Educational  Testing  Service 
Princeton,  NJ  08541-0001 

Dr.  Robert  L.  Linn 
Campus  Box  249 
University  of  Colorado 
Boulder,  CO  80309-0249 
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Dr.  Robert  Lockman 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  Frederic  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  George  B.  Macreadv 
Department  of  Measurement 
Statistics  &  Evaluation 
College  of  Education 
University  of  Maryland 
College  Park,  MD  20742 

Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  08451 


Dr .  William  Montague 

NPRDC  Code  13 

San  Diego,  CA  92152-6300 

Ms.  Kathleen  Moreno 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Headquarters  Marine  Corps 
Code  MP1-20 
Washington,  DC  20380 

Dr.  W.  Alan  Nicewander 
University  of  Oklahoma 
Department  of  Psychology 
Norman,  OK  73071 

Deputy  Technical  Director 

NPRDC  Code  01A 

San  Dieyo,  CA  92152-6800 


Dr.  James  R.  McBride 
The  Psychological  Corporation 
1250  Sixth  Avenue 
San  D i ego ,  CA  92101 

Dr.  Clarence  C.  McCormick 
HQ,  USMEPCOM/MEPCT 
2500  Green  Bay  Road 
North  Chicago,  IL  60064 

Dr .  Robert  McK i n I ey 
Educational  Testing  Service 
1 6— T 

Princeton,  NJ  0S541 

Dr.  James  McMichael 
Technical  0 i recto r 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  Barbara  Means 
SRI  International 
333  Ravenswood  Avenue 
Menlo  Park,  CA  94025 

Dr.  Robert  Mislevy 
Educational  Testing  Service 
Princeton,  NJ  08541 


Director,  Training  Laboratory, 
NPRDC  (Code  05) 

San  Diego,  CA  92152-6800 

Director,  Manpower  and  Personnel 
Laboratory , 

NPRDC  (Code  06) 

San  Diego,  CA  92152-6800 

Director,  Human  Factors 

&  Organizational  Systems  Lab, 
NPRDC  (Code  07) 

Sari  Diego,  CA  92152-6800 

Library,  NPRDC 
Code  P201L 

San  Diego,  CA  92152-6800 

Commanding  Officer, 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

Dr.  Harold  F.  O'Neil,  Jr. 

School  of  Education  -  WPH  801 
Department  of  Educational 
Psychology  &  Technology 
University  of  Southern  California 
Los  Angeles,  CA  90089-0031 
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Or.  James  B.  Olsen 
WICAT  Systems 
1875  South  State  Street 
Orem,  UT  84058 

Office  of  Naval  Research, 

Code  1142CS 
800  N.  Quincy  Street 
Arlington,  VA  22217-5000 
<6  Copies) 

Office  of  Naval  Research, 

Code  125 

800  N.  Quincy  Street 
Arlington,  VA  22217-5000 

Assistant  for  MPT  Research, 
Development  and  Studies 
OP  0tB7 

Washington,  DC  20370 

Dr.  Judith  Orasanu 
Basic  Research  Office 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Jesse  Qrlansky 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  St. 
Alexandria,  VA  22311 

Dr.  Randolph  Park 
Army  Research  Institute 
5C01  Eisenhower  Blvd. 
Alexandria,  VA  22333 

Wayne  M.  Patience 
American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Circle,  MW 
Washington,  DC  20036 

Dr.  James  Paulson 
Department  of  Psychology 
Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 


Department  of  Operations  Research, 
Naval  Postgraduate  School 
Monterey,  CA  93940 

Dr.  Mark  D.  Reckase 
ACT 

P.  0.  Box  168 
I owa  City,  I A  52243 

Dr.  Malcolm  Ree 
AFHRL/MOA 

Brooks  APB,  CX  73235 

Dr.  Barry  Riegelhaupt 
HumRRO 

1100  South  Washington  Street 
Alexandria,  VA  22314 

Dr.  Carl  Ross 
CNET-PDCD 
Building  90 

Great  Lakes  NTC,  IL  30088 
Dr.  J.  Ryan 

Department  of  Education 
University  of  South  Carolina 
Co  I umb i a ,  SC  29208 

Dr.  Fumiko  Samejima 
Department  of  Psychology 
University  of  Tennessee 
310B  Austin  Peay  Blda. 

Knoxvi lie,  IN  37916-0900 

Mr.  Drew  Sands 

NPRDC  Code  62 

San  Diego,  CA  92152-6800 

Lowe ! '  Schoer 

Psychological  &  Quantitative 
Foundat i ons 
College  of  Education 
University  of  Iowa 
Iowa  City,  I A  52242 

Dr.  Mary  Schratz 

Navy  Personnel  R&D  Center 

San  Dieoo.  CA  92152-6300 


Dept,  of  Administrative  Sciences 
Code  54 

Naval  Postnradua  t.e  School 
Monterey,  CA  93943-5026 


Dr .  Dan  Segal  I 

Navv  Personnel  Rf»D  Center 

San  Diego,  CA  92152 
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Cr.  W.  Steve  Sellm3n 
0ASD<MRA*L> 

28289  The  Pentagon 
Washington,  DC  20301 

Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuma-Ka i gan 
Fuj i sawa  251 
JAPAN 

Dr.  Wi I  I i am  Sims 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  H.  Wallace  Sinaiko 
Manpower  Research 

and  Advisory  Services 
Smithsonian  Institution 
801  North  Pitt  Street,  Suite  120 
Alexandria,  VA  22314-1713 

Dr.  Richard  E.  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

Dr.  Richard  C.  Sorensen 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  Paul  Speckman 
University  of  Missouri 
Department  of  Statistics 
Co  I umb i a,  MO  65201 

Dr.  Judy  Spray 
ACT 

P.0.  Box  168 
Iowa  City,  IA  52243 

Dr.  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  William  Stout 
University  of  Illinois 
Department  of  Statistics 
101  I  II  ini  Hall 
725  South  Wright  St. 

Champaign,  IL  61820 


Or.  Hariharan  Swaminathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 

Mr.  Brad  Sympson 

Navy  Personnel  R&D  Center 

Code-62 

San  Diego,  CA  92152-6800 

Dr.  John  Tangney 
AFOSR/NL ,  Bldg.  410 
Bolling  AFB,  DC  20332-6448 

Dr.  Kikumi  Tatsuoka 
CERL 

252  Engineering  Research 
Laboratory 

103  S.  Mathews  Avenue 
Urbana,  IL  61801 

Dr.  Maurice  Tatsuoka 
220  Educat ion  Bldg 
1310  S.  Sixth  St. 

Champaign,  IL  61820 

Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

Mr.  Gary  Thomasson 
University  of  Illinois 
Educational  Psychologv 
Champaign,  IL  61820 

Dr.  Robert  Tsutakawa 
University  of  Missouri 
Department  of  Statistics 
222  Math.  Sciences  Bldg. 
Columbia,  MO  65211 

D-.  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  Street 
Champaign,  IL  61820 
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Or.  Vern  W.  Urry 
Personnel  R&D  Center 
Office  of  Personnel  Management 
1900  E.  Street,  NW 
Washington,  DC  20415 

Dr.  David  Vale 
Assessment  Systems  Corp. 

2233  University  Avenue 
Su i te  440 

St.  Paul ,  MN  55114 

Dr.  Frank  L.  Vicino 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-8800 

Dr.  Howard  Wainer 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Ming-Mei  Wang 
Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  IA  52242 

Dr.  Thomas  A.  Warm 
Coast  Guard  Institute 
P.  U.  Substation  18 
Oklahoma  City,  OK  73169 

Dr.  Brian  Waters 
HumRRO 

12908  Argyle  Circle 
Alexandria,  VA  22314 

Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Ronald  A.  Weitzman 
Box  146 

Carmel,  CA  93921 

Major  John  Welsh 
AFHRL/MOAN 

Brooks  AFB,  TX  78223 


Dr.  Douglas  Wetzel 
Code  51 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6300 

Dr .  Rand  R .  Wilcox 
University  of  Southern 
Cal i f  orn i a 

Department  of  Psychology 
Los  Angeles,  CA  90039-1061 

German  Military  Representative 
AT  IN:  Wolfgang  Wildgrube 
Stre i tkraef  teamt 
D-5300  Bonn  2 

4000  Brandywine  Street,  NW 
Washington,  DC  20016 

Dr.  Bruce  Willi ams 
Department  of  Educational 
Psycho  I ogy 

Un i vers i ty  of  1 1  I i no i s 
Urbana,  III  61801 

Dr .  Hilda  Wing 
NRC  MH-176 

2101  Constitution  Ave. 
Washington,  DC  20418 

Dr.  Martin  F.  Wiskoff 
Defense  Manpower  Data  Center 
550  Camino  El  Estero 
Suite  200 

Monterey,  CA  93943-3231 

Mr.  John  H.  Wolfe 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152-6800 

Dr.  George  Wong 
Biostatistics  Laboratory 
Memorial  S I oan-Ketter i ng 
Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

Dr.  Wallace  Wult'eck,  III 
Navy  Personnel  R&D  Center 
Code  51 

San  Diego,  CA  92152-6800 
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Or.  Kentaro  Yamamoto 
03-T 

Educational  Testing  Service 
Rosedale  Road 
Princeton,  NJ  08541 

Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 

Dr.  Joseph  L.  Young 
National  Science  Foundation 
Room  320 

1800  G  Street,  N.W, 
Washington,  DC  20550 

Mr.  Anthony  R.  Zara 
National  Council  of  State 
Boards  of  Nursing,  Inc. 
625  North  Michigan  Avenue 
Suite  1544 
Chicago,  IL  60611 

Dr.  Peter  Stoloff 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 
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